Many internet users discover and interact with internet documents using search queries. For example, a user may search for websites, images, videos, and other internet documents by submitting a query to a search engine. A search engine may crawl the web to fetch millions of documents that may be used as search results (e.g., a web crawler may fetch images, articles, and/or other documents from websites). It follows that the quality of search results provided by the search engine may be based upon the desirability of the fetched documents that the search engine retrieves. Unfortunately, undesirable documents may be fetched from document providers. In one example, a website may perform a soft block by silently redirecting a web crawler to a junk page because the website may perceive the web crawler as potentially rude. In another example, a website may be down for maintenance or unexpected downtime, and thus a web crawler may be directed to a maintenance page. The junk page and/or maintenance page may be undesirable documents to provide as search engine results.